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ABSTRACT 



Measuring school and system performance presents a major 
challenge and opportunity for governments. The public is demanding stringent 
quality reviews to ensure that the multi-billion dollar education investment 
has the greatest possible impact on student learning. Current methods to 
evaluate system performance do not do a good job of demonstrating how well 
students achieve intended goals or the impact of program spending. Effective 
accountability systems will embody a continuous cycle of assessment, 
analysis, and action. Agreement on the indicators and tools to assess them is 
only a first step. Attention must be paid to policy options for developing 
effective strategies to focus the system on results and building school 
capacity for improvement. Promising new strategies across North America are 
examined, including setting targets, publishing school profiles, incentive 
plans and sanctions, school performance contracts, and new personnel 
practices. Truly effective accountability systems will be those that change 
the internal dynamics of our schools and address the will and capacity of 
schools to use performance data to improve their practices and engage the 
whole community in supporting improved student learning. (Contains 29 
references.) (Author/SLD) 
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Synopsis: Measuring school and system performance presents a major challenge and opportunity 
for governments. The public is demanding stringent quality reviews to ensure the multi-billion 
dollar education investment has the greatest possible impact on student learning. Current methods 
to evaluate system performance do not do a good job of demonstrating how well students achieve 
intended goals or the impact of program spending. Effective accountability systems will embody a 
continuous cycle of assessment, analysis and action. Agreement on the indicators and tools to 
assess them is only the first step. Attention must be paid to policy options for developing effective 
strategies to focus the system on results and building school capacity for improvement. The author 
examines promising new strategies across North America, including setting targets, publishing 
school profiles, incentive plans and sanctions, school performance contracts and new personnel 
practices. The conclusion suggests that truly effective accountability systems will be those that 
change the internal dynamics of our schools and address the will and capacity of schools to use 
performance data to improve their practices and engage the whole community in supporting 
improved student learning. 



Measuring school and system performance presents a major challenge for governments. Ontario’s 
most recent Royal Commission on Learning sharply reflected this reality in its report: 



“ One complaint we heard repeatedly was that the public education system 
no longer seems responsible to the public... There exists widespread unease 
that schools have become kingdoms unto themselves, with little need to 
report to parents or the world at large what they are doing with our kids 
and whether or not they are doing it successfully. ” 



Canada is not alone. Education Week’s comprehensive 1997 report card on the condition of public 
education in the United States noted a troubling lack of useful hard data on performance: “Public 
education is a vast enterprise.... Its success is clearly linked to the welfare of the nation and the 
future of our children. Yet we do not know in any but the crudest way, how well our education 
system is performing.” 
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In the past three years, however, there has been a marked general trend across North America 
towards increased assessment of school system performance. Government audits have begun 
insisting on measures of value for education tax dollars invested. Although this reflects a sound 
reading of the public mood, it requires a new culture of evaluation in our school systems. 

Much of the education establishment still harbors an intense distrust of achievement data. 
(Schmoker, 1996) Many educators and school district officials argue that learning cannot be 
measured accurately, that testing restricts educational experiences and penalizes the disadvantaged. 
Included among the key objections by the education community to standardized assessment 
programs are: inducing ‘teaching to the test’, narrowing the curriculum, tapping only lower-order 
thinking skills, administrative cost, hurtful to low achievers, and test biases unfair to minorities. 
While many of these concerns can be disproved or alleviated through more sophisticated 
assessment practices (Phelps, 1999), they remain a common obstacle to system measurement. 

This resistance may be illustrated through the reaction to the B.C. Office of the Comptroller- 
General report released in 1997. The report found the education ministry and the school system 
“need to improve measurement and the reporting of how well students achieve intended goals.” A 
countering brief to the government submitted by the B.C. Teachers’ Federation insisted that 
accountability must only be measured in terms of “opportunity to learn”, not in measurement of 
learning outcomes. The BCTF requested elimination of the province-wide learning assessment 
program newly employed to track both school and program performance over time. In spring of 
1999, the Minister of Education announced school-level results would not be released until 
concerns could be resolved. 

Yet school assessment holds promise of succeeding where other reforms have failed in raising 
student performance. As Schmoker notes, “ Goals rightly defined and pursued, are the most 
crucial element in any school system that hopes to get better results.” (Schmoker, 1997) Although 
the classroom has proved remarkably well-insulated from most government education policies, 
using student scores on prescribed tests to make judgments about the quality of education provided 
in a particular school or district has evoked changes in what happens in the classroom. (Mumane, 
Levy, 1996) Recent national comparisons of standardized achievement results have demonstrated 
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that jurisdictions in both the US and Canada implementing the strongest assessment and public 
accountability programs have realized the largest gains in student achievement. (Grissmer 1 998, 
NAEP 1999, CMEC, 1999) 

Governments are now seeking the best tools and measures to improve learning outcomes. As they 
develop frameworks to provide useful data for guiding educational decisions and program 
investments, they are asking: 

• What are our priorities and targets for future learning gains? 

• How can we cause schools to focus on goal-setting for improved learning? 

• What is the evidence that a program significantly improves student learning? 

• How can funds be reallocated to support what works? 

Linking Assessment to Accountability 

Finding appropriate school evaluation systems means clarifying the relationship between 
assessment and accountability. The Annenberg Institute suggests accountability is the obligation or 
responsibility to demonstrate effectiveness, and that it has four components: 

1. the development of standards that define desired outcomes 

2. the evaluation or assessment of progress towards those outcomes 

3. analysis of the resulting data 

4. corrective action based on that knowledge 

Such a process goes beyond mere assessment - the gathering, organizing and reporting of 
information. Accountability uses data to make judgments about where we need to adjust behaviors 
in a constant cycle of assessment, analysis and action to improve future performance. 

Joan Green, the chief of Ontario’ s Education Quality and Accountability Office, made this clear in 
the first round of that province’s mandatory provincial testing of skills in reading and math in 
1997: “We believe large-scale assessment can contribute to positive educational change when it 
engages educators, parents and students in thought and discussion about what takes place in the 
classroom. We are committed to a cycle of assessing and reporting that leads to action.” 
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Tests are not Enough 

Testing reveals what students know and are able to do in core competencies we expect them to 
master. Achievement scores alone, however, have limited power to improve student learning. It is 
only when school communities have the will and capacity to reflect upon a range data and utilize it 
for future performance planning, that assessment will be valued. 

But test scores alone do not provide sufficient information on school success. A broader range of 
indicators is needed, including parent and student satisfaction levels, attendance rates, school 
leadership, community involvement, and valuable student learning which cannot be captured by 
exams. Schools will need to develop reliable tools for measuring their own school-specific goals in 
other than core learning. Complimentary use of internal and external evaluation can point schools 
to the path for necessary change to improve student outcomes. (Marchesi, 1998) This total 
performance feedback provides useful guidance for practitioners. 

As Michael Fullan observes, “The way to deal with potential misuse of student performance data is 
to become assessment literate. Schools put themselves in the driver’s seat when they invest in 
professional development and collaborative cultures that focus on student learning and associated 
improvements in instructional practices.” 

Successful schools are now grappling head-on with building their capacity for improvement. We 
know from the research findings that this requires developing deeply embedded school practices 
and habits that strengthen the focus on results. The challenge is how to “re-culture” every school to 
achieve this. 

Building Capacity to Respond 

If we wish to encourage schools to use results to improve student success, the system must be 
designed in a way that both identifies problem areas and enables those responsible to respond. 

Early accountability systems often failed because they attempted to employ top-down approaches 
which ignored the high-involvement principles at the local level necessary to impact student 
learning. 
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School improvement is a highly complex and collaborative exercise. No single magic solution or 
simplistic measure will work for all schools. The task of educating very diverse learners to much 
high standards of learning in a world of with fast-changing educational demands will require more 
responsive schools than present bureaucracies allow. High-performance schools require a high 
level of autonomy and flexibility at the school site. (Darling-Hammond, 1 996). Engagement of the 
total individual school community in designing solutions and responses to their unique learning 
environment must be possible in any accountability plan. 

Creating Incentives that Work 

Some practices are emerging as policymakers attempt to create a climate in which all schools can 
become centers of excellence. The question to ask of each strategy will be. How well does it raise 
student achievement over time? We are just now beginning to accumulate some early evidence. 

Setting Targets 

Governments can influence system performance by analyzing strengths and weaknesses and 
identifying improvement targets. A few Canadian provinces such as Alberta, Newfoundland and 
P.E.I. have developed multi-year education business plans. The best established is Alberta’s 
Education’s Three-year Business Plan now in its second cycle, which publishes comparative 
progress with previous years on a wide variety of indicators, and announces specific provincial 
targets for the following years. Schools and boards are required to develop annual education plans 
aligned with the targets and report on their progress. This act of quantifying goals has focused 
attention, resources and efforts on the result. In national assessments, Alberta consistently rates at 
or near the top in student achievement. 

Connecticut produced the largest reading gains on the National Assessment of Educational 
Progress (NAEP) between 1992 and 1998, with the percentage of grade four students at or above 
proficiency levels rising from 34% to 46%. This was achieved through “a deliberate attempt by 
state policymakers to improve student literacy skills through rigorous assessment and targeted aid.” 
(Education Week, March 10, 1999). Efforts to meet state reading goals included administration of 
Connecticut Mastery tests with annual targets, dedicating $20 million a year to urban reading 
intervention programs and 20% of priority districts’ funding to early literacy programs, and 
requiring three-year plans by districts for raising reading performance. 
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Publishing School Profiles 

Thirty-six U.S states now require all schools to publish annual school progress report cards. Some 
districts make school profiles available on the web. Ontario’s North York School District publishes 
a detailed annual profile of all schools in a binder for parents. A four-page report card on each 
school provides test results, demographics, special programs and school improvement plans, giving 
parents assessment information in a meaningful context. The Calgary Public School Board recently 
reversed its long-standing policy of opposition to published school-level achievement data in the 
face of evidence that careful use of this information for school-improvement planning and 
reallocation of resources according to ‘what works’ was successfully used by the Calgary Catholic 
School Board to raise performance in their schools. (Calgary Herald, January 1999) 

In Texas, an aggressive program over the past decade of annual assessment and published school 
profiles has produced achievement state-wide gains year after year, which would appear to be 
corroborated by gains well above the national average on NAEP. Observers of Texas also report 
(Phelps, 1999 p.5): 

□ A greater focus on academic learning 

□ A' culture of high expectations and enthusiasm for reaching standards 

□ Generous and immediate remediation efforts towards poorly-performing students 

□ Greater interest among teachers in academic strategies and more cooperation with each other 

□ Quicker feedback for school faculty on which instructional strategies work best 

Practitioners’ resistance to the release of school performance data is fueled by blatant examples of 
misuse of data, such as the raw ranking of schools, and the fear that such information is harmful to 
weaker schools. The fundamental principle underlying the use of data to spur performance, 
however, is that each school competes against its own previous results. This value-added format 
can be helpful to all schools, even those which serve disadvantaged students. Schmoker (1996, 97) 
finds ample evidence to suggest that direct scrutiny of individual school results year on year is the 
most effective way to narrow the gap between high and low income students. Analysis of the most 
recent National Assessment of Educational Progress (NAEP) supports the conclusion that low 
income and minority groups are making gains through rigorous attention to assessment results, 
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although they are still performing considerably below their more advantaged peers. Two states with 
strong assessment programs and larger than average disadvantaged populations, North Carolina 
and Texas have made the biggest gains. 

Many researchers point out the need to build a culture of assessment literacy amongst both public 
and educators. One recent trend is the development of technically proficient experts at state and 
district levels to disaggregate and assist in interpreting data for a variety of audiences. Some 
jurisdictions such as Virginia, Massachusetts, Delaware and Washington are paying increasingly 
careful attention to education of the public and the media about the meaning of assessments before 
the annual information is released. (Education Week, 1 999) 

School Based Processes for Examining Achievement 

In some provinces, school councils are mandated to develop and oversee school improvement 
plans. Nova Scotia’s school councils must “prepare an annual accountability report describing 
school improvement activities undertaken during the year and the specific results achieved.” Some 
systems such as British Columbia require a cyclical school accreditation process which includes 
self and peer-review components. New Mexico and Rhode Island are among states which have 
combined statistical profiles of a school’s performance with on-site visits by teams of educators 
and others to examine school practices. In some jurisdictions, internal school evaluations are 
combined with an external quality review process by a government agency, such as the Texas 
Education Agency. 

New formats for accessing quality professional development are key to building school capacity to 
respond effectively to achievement results. Policymakers suggest that without making this a 
priority, standards and accountability efforts alone will not succeed. Professional development 
must be aligned with standards and assessment, focused on student achievement, flexible and 
responsive to individual school needs, accessible, convenient and adequately funded. (Watts, 1998) 
Finding time for professional training and school improvement planning is one of the most serious 
obstacles. Alternatives such as lengthening the school year and intensive summer sessions with 
follow-up clinics on weekends are being explored. 
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Performance Incentive Plans 

In Dallas, Texas teacher and principal performance rewards are tied to learning results. Each year, 
$2.4 million is awarded to school staffs where ‘value-added’ in student learning outcomes is 
highest. Schools compete against their own previous scores, not against arbitrary norms. In 
Charlotte-Mecklenburg, North Carolina, a school evaluation plan combines achievement results- 
based evaluations with cash bonus awards. Fourteen states and the province of Alberta now 
provide monetary rewards to schools or districts meeting their individual improvement targets on a 
variety of measures. 

The premise of such cooperative performance incentive plans (CPI) is that when an entire school 
community works together to raise student performance over its own previous benchmarks, there 
are more positive results than with plans that merely reward individual efforts. CPI plans may take 
3-5 years to fully implement. Further evaluation is needed to determine if they have been more 
effective than earlier individual merit pay plans. What is known, is that some states such as 
Kentucky, Texas and North Carolina using CPI approaches for five or more years are 
demonstrating significant gains. 

Conditions leading to success of CPI programs have been defined as open access to information, 
site-based decision making, a support system that helps schools interpret and assess data, 
persistence and a significant commitment of resources. (Richards, 1993) Perceived fairness is 
critical to a CPI plan’s acceptance, with care taken to ensure a level playing field and that progress 
for each unit is measured on a value-added basis. Districts that have successfully implemented CPI 
programs have sought cooperative input from the field at all phases of design and implementation 
and offer rewards commensurate with the level of effort required to attain. (Kelley, 1996). 

School Performance Contracts 

Districts such as Boston, Philadelphia and Baltimore have established pilot public schools under 
specific performance contracts to fill unmet educational needs in the district. Baltimore, for 
example, sought proposals geared to under-served and disadvantaged students. By a joint 
agreement between the union and the district, these contract schools are given wide latitude from 
district and contract regulations to encourage innovative efforts to improve student achievement. 
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Public charter schools expand this principle of autonomy for accountability. Over 1200 of these 
schools are operating as independent legal entities, and may continue to exist only if students 
demonstrate the results and educational goals specified in their charter. Comprehensive research 
now underway in (U.S. Department of Education, 1998 and others) and Canada (Bosetti, 1998) 
will determine the success of the charter movement in raising student achievement. 

Personnel Practices Tied to Achievement: 

There is a growing understanding of the need to place a high priority on the quality of instruction in 
the classroom, rendered next-to-impossible by many collective agreements. A groundbreaking 1997 
Seattle Public Schools teacher contract alters teacher evaluation and hiring provisions. It ties 
teacher assignment to skills required by the school and links teacher evaluations to gains in student 
achievement. In Texas, an appraisal system begun in 1997-98 requires districts to take into 
account student performance when evaluating teachers. Tennessee’s Value-Added Assessment 
System generates confidential information on the performance of each teacher’s students which 
principals may use in recommending professional development. The diagnostic information 
provided has underscored the “dramatic effect skilled teachers have on performance.” (Quality 
Counts ’99, p. 2) 

In other personnel quality assurance moves, Oregon has eliminated teacher and principal tenure. 
Washington DC and Florida have reduced probationary periods to 90 days, speeding up the time it 
takes to remove weak teachers. Other states have provided additional resources to districts to 
retrain or remove administrators whose schools fail to show improvement, and to help school 
administrators assess, re-train or dismiss inadequate teachers. Some states have implemented 
mentoring programs. Thirty states have placed greater emphasis on rewarding advanced 
professional certification and professional development. In 1998, the National Education 
Association adopted resolutions supporting effective monitoring of teacher quality. 

This fits with forecasts of increasing decentralization of schools and teaching in the 21st century. 
Some education labor experts predict that when teachers are hired by individual schools on 
contracts reflecting the goals of the school, teacher union focus will shift from job control, work 
rules and uniformity to organizing around professional competence, quality assurance and 
increased productivity. (Kerchner, Koppich, and Weeres, 1997) Carefully designed competency- 
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based pay to reward skills, combined with school team awards for collaborative achievement of 
performance goals, will provide the incentives teachers need to develop the skills to perform to 
heightened expectations. (Kelley, 1996). Evidence of the impact on student achievement from new 
personnel practices remains to be gathered. 

Interventions in Failing Schools 

Some policymakers are intervening where assessment shows persistent failure to improve student 
learning. When Chicago Superintendent Paul Valias placed over 100 schools on academic 
probation in 1997, test results show marked improvement across the district. The largest gains 
were made by the poorest performing students, where schools on probation changed instructional 
methods and extended the school day. North Carolina reports 15 schools designated for ‘mandatory 
assistance’ in 1996-97 finished the 1997-98 year with improvements of more than 10%. 

Twenty-three U.S. states have ‘academic bankruptcy’ provisions, allowing them to intervene 
directly in persistently low-performing schools. These measures range from additional training and 
resources, to replacing district or school leadership, replacing school staff, or closing the school. In 
practice, few states have been willing to use the severest sanctions. Evidence to date on the 
effectiveness of drastic intervention efforts is largely anecdotal and incomplete at this point. 

What is clear is that chronically low-performing schools usually have limited capacity on their own 
to make the changes necessary to improve achievement. Such schools require not only pressure but 
support. More attention is being paid to district and state roles identifying failing schools, 
providing technical assistance and additional resources, and evoking a collaborative process 
involving all stakeholders to redesign these schools. A key policy element in North Carolina and 
Texas, both recognized for producing rapid achievement gains, has been the shifting of resources to 
schools with more disadvantaged students. (Grissmer, 1999) 

Conclusion 

There are now many strategies available for policymakers to link assessment and accountability to 
improve student learning. Truly effective assessment systems will change the internal dynamics of 
our schools and nurture the leadership and flexibility required to design and implement pathways to 
success for every school. 
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